Multilayer Perceptrons based on Fuzzy Flip- Flops

The concept of fuzzy flip-flop was introduced in the middle of 1980’s by Hirota (with his students). The Hirota Lab recognized the essential importance of the concept of a fuzzy extension of a sequential circuit and the notion of fuzzy memory. From this point of view they proposed alternatives for “fuzzifying” digital flip-flops. The starting elementary digital units were the binary J-K flipflops. Their definitive equation was used both in the minimal disjunctive and conjunctive forms. As fuzzy connectives do not satisfy all Boolean axioms the fuzzy equivalents of these equations resulted in two non equivalent definitions, “reset and set type” fuzzy flip-flops (F 3 ), using the concepts of fuzzy negation, tnorm and t-conorm operations. Hirota et al. recognized that the reset and set equations cannot be easily used as elements of memory module because of their asymmetrical nature. In their 1988’s paper Ozawa, Hirota and Kóczy proposed a unified form of the fuzzy J-K flip-flop characteristic equation involving the reset and set characteristics, based on min-max and algebraic norms. A few years later the hardware implementation of these fuzzy flip-flop circuits in discrete and continuous mode was presented by the Hirota Lab. The hardware implementation of algebraic norm based F 3 pointed out the methodology benefit resulting from the deep mathematical relation of fuzzy systems and neural networks. The fuzzy flipflop was proposed as basic unit in fuzzy register circuits. The multilayer perceptron (MLP) is a widely used artificial neural network type. The MLP as function approximator is the object of study in different engineering applications as well as applied mathematics and computer science. We introduced Fuzzy Flip-Flop based Neural Networks (FNN), a multilayer perceptron based on fuzzy flip-flops. The proposed network is a structure consisting of the same type of fuzzy flip-flops. In the FNN the neurons have been substituted by fuzzy J-K flip-flops with feedback (in reality, a type of combinational circuit) and fuzzy D flip-flops derived from the fuzzy J-K F 3 based on frequently used fuzzy operations (e.g. algebraic, Łukasiewicz, Yager, Dombi, Hamacher, Frank and Dubois-Prade norms). Several types of fuzzy J-K and D flip-flop based on various fuzzy operations and standard complementation have been proposed. We determined their characteristic equations, illustrating and comparing their properties. It was shown that broadly they may be classified into two groups, one presenting quasi s-shape transfer characteristics and the rest having non-sigmoid character. The construction of a fuzzy neuron unit from fuzzy J-K and D flip-flops was proposed indicating a possible connection of this fuzzy unit with the artificial neuron, the basic component of neural networks. The main idea is to have such a fuzzy neural network structure which can be implemented simply in hardware. The fuzzy norms, furthermore simple fuzzy flipflop characteristic equations are not complicated in hardware realization. We proposed the FNN as a neural network with general-purpose hardware which could be more user-friendly; it is not bound to algorithmic a-priori-assumptions and therefore offers high flexibility. We defined a pair of new fuzzy intersection and union. The proposed connectives, called Trigonometric norms, consist of simple combinations of trigonometric functions. The basic motivation for constructing new norms was to have fuzzy flip-flops with sigmoid transfer characteristics in some particular cases. We compared the function approximation capability of FNNs with one and two hidden layers based on Dombi, Łukasiewicz and new Trigonometric norms. A special combination of the Levenberg-Marquardt (LM) method with the Bacterial Evolutionary Algorithm (BEA), the Bacterial Memetic Algorithm with Modified Operator Execution Order (BMAM) was proposed and applied for FNNs variables optimization and training. We showed that the FNNs function approximation accuracy depends not only on the network structures and parameters selected, on the hidden layer numbers, and on the hidden units, but is strongly influenced by the fuzzy flip-flop and operator type, furthermore their parameter values and the learning algorithm. Comparing the simulation results, we concluded, that Łukasiewicz type fuzzy D flip-flop neurons based NNs with two hidden layers performed best; they had the best generalization capability. Acknowledgments This work is partially supported by Óbuda University Grants, the project TÁMOP 421B, a Széchenyi István University Main Research Direction Grant, further the National Scientific Research Fund Grants OTKA K 75711, and K 105529.